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Introduction 

A critical issue in the optimization of systems is that of 
the design of the part, component or system for 
manufacture. The system engineering function typically 
finds an acceptable but theoretical design solution with 
little up-front attention being paid to the "real-world" 
effects caused by manufacturing and operational 
variabilities. Alternatively when system design attention 
is paid to such variabilities, it is usually via a "worst- 
case" design process which can only further complicate 
the manufacturing process and raise costs. To be truly 
effective in the near future of small-lot flexible system 
manufacturing, systems engineering must not only 
optimize against the effects of changing manufacturing 
variabilities, but the overall systems design and 
manufacturing processes must be woven more closely 
together so as to permit routine first pass success [I]. 

As a resutt of the above issues, the importance of 
running early statistically based system level design 
simulations has become particularly critical in this period 
of increasingly flexible manufacturing processes, 
defense conversions and dual-use technologies. 

Statistical Simulation Methodology 

In systems engineering, physical simulators are used to 
describe relationships between the system's design and 
the manufacturing processes in terms of the system's 
design characteristics. The goal is to permit the system 
to be designed and manufactured correctly and 
efficiently. Effects of uncontrollable variation in 
manufacturing and operational parameters can, 
however, lead to variations in the end result. For 
instance, it may be that the system characteristics are 
inordinately sensitive to slight variations in the 
manufacturing process consequently resulting in 
potential reduction in yield or percentage of 
marketable systems produced. Obviously, 
understanding and controlling the sensitivity of a 
system's characteristics can be critical to the concept of 
flexible quick turn around systems design while allowing 
more reliable systems and increasing the overall quality 
of the system. 1 
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Monte Carlo techniques have often been used to provide 
simple approaches to the statistical simulation of 
engineering problems [2]. However while Monte Carlo 
based methods have the theoretical capability for 
statistical accuracy, they suffer from a number of real- 
world difficulties. Large numbers of simulation runs are 
required to obtain any reasonable degree of statistical 
accuracy. Furthermore, Monte Carlo approaches do not 
provide other critical engineering and manufacturing 
information such as how the system response varies as 
a function of input variables, or how to optimize this 
response. 

Design of Experiments (DoE) is an alternate statistical 
optimization methodology long used in process 
development [3J. It provides a systematic way of 
characterizing the variabilities of a process while 
minimizing the number of replications needed to do so. 
However, this statistical methodology has not been used 
in design of flexible manufacturing processes for several 
reasons. The primary reason is that it has usually been 
laborious and practiced only by experienced 
statisticians. Although design of experiments can be 
used to estimate very useful statistical parameters such 
as the mean and standard deviation of the output 
response, it does involve complex but straight forward 
mathematical manipulations that can best only be 
applied by computer. Recently, Design of Experiments 
has been used to accomplish statistical simulations of 
integrated circuit process development [4,5]. Our 
solution thus involves the development of a computer 
program to guide system design engineers through this 
complex mathematical sequence toward system 
optimization. 

Another reason why design of experiments methodology 
is not typically used to full advantage in system 
engineering optimizations involves the practical 
difficulties encountered with simulators. Most simulators 
require the user to possess in depth knowledge of both 
the manufacturing processes and the specific workings 
of the simulator packages themselves. Our solution to 
this problem involves a user-friendly input system where 
a user easily enters the specifics of the system to be 
manufactured. The simulations are then automatically 
set up and run with software selection of proper input 



variations as dictated by the embedded design of 

experiments theory. 

STADIUM 

The software program being discussed in this paper is 
called STADIUM and was originally conceived and 
developed at Florida Institute of Technology. The major 
functional blocks of STADIUM are shown in Figure 1 
which also shows how they form an integrated 
capability. STADIUM can employ almost any valid 
simulator of interest and achieve correct statistical 
predictions of the results of the manufacturing 
operations[6,7J. 

STADIUM is primarily employed in predicting the 
variation of system level parameters due to expected 
incidental variations encountered during the phases of 
manufacturing and use. The statistical design of 
experiments methodology is typically accomplished 
using an internal customized statistical package. In 
each case, it is automatically accomplished in 
conjunction with relevant component, subsystem and 
system level simulators to allow the proper design of the 
statistical experiment. 




Figure 1: The STADIUM Conceptual Architecture 

This statistical experiment is then automatically run 
using component, subsystem and system level 
simulators relevant to the problem at hand. The result of 
this computer designed statistical experiment is 
analyzed under user supervision to provide the precise 
statistical description of system performance as a 
function of those manufacturing parameters found to be 
of significance. These analyzed statistical results can 
now be reliably used by system engineers as an aid in 
optimizing system performance and quality, reducing 
manufacturing costs and making life-cycle trade-offs. 

STADIUM utilizes a graphical menu driven X-window 
user interface to allow for specification of known 
manufacturing variables and operation parameters. 
These processes are independent of the simulators to 
be employed and thus the system engineer does not 
need to be overly familiar with the interface 
characteristics of the variety of simulators themselves. 
STADIUM then uses this data via automated design of 
experiments techniques to conduct a batch run of 



simulations - the results of which are automatically 
analyzed for performance prediction. The software 
architecture of the STADIUM package facilitates the 
ready integration of alternate simulators from a variety of 
vendors. Thus STADIUM also lends itself to the flexible 
system engineering design concept by itself being 
readily readapted to newly emerging variations of the 
system and manufacturing process. 

Application to System Optimization 

Some of the worlds most complex and costly electronics 
products have been developed for the military. Now in a 
different time, the emphasis is swinging to rapid flexible 
design and manufacture of systems on demand. This is 
of course layered onto the already difficult problems of 
developing extremely complex and sophisticated 
systems. Earlier approaches toward handling the design 
complexity of the system, i.e. a methodology of systems 
engineering centering around a series of checks and 
balances of technical specification, is now inadequate for 
the future, in yesterday's system engineering, concepts 
of statistical manufacturing variability, while understood, 
were for the most part ignored in the drive for technical 
excellence. Now, escalating system costs and reduced 
budgets lead to the serious consideration of flexible 
system optimization to the task at hand on demand. 
These needs have become increasingly serious in the 
already current environment of defense conversion and 
dual-use technology. Consequently, we present here an 
example of how the STADIUM statistical methodology 
can now form a critical component within the domain of 
system optimization. 

Large complex systems are usually subject to a variety 
of requirement demands including those of technical 
performance, usability and reliability. This has led to the 
development of specialized system design simulators. 
In contradiction to other simulators, system design 
simulators do not simply predict physical parameters. 
They are typically more geared to satisfaction of tasking; 
i.e., how well can insertion of a system into a larger and 
enveloping environment improve the overall tasking. 
Thus system simulators must simulate portions of the 
tasking environment as well as that of the system under 
design. They are however not generally responsive to 
determining the impact of variabilities within the 
manufacturing environment. 

In the area of night vision system design, the FL1R90/ 
ACQUIRE [8] (and subsequent FUR 92/ACQUlRE) 
simulator is the standard in task oriented FLIR system 
simulation. It uses a simulated "standard" human 
observer and a rigorous FLIR system simulation to allow 
design trade-offs within an overall FLIR tasking. FLIR 
systems developed to the component specifications 
satisfying the FLIR90/ACQU1RE simulation will be 
generally accepted by a customer without need for 
conventional field trials. 



A particular difficulty with the use of system design 
simulators, such as FLIR90/ACQUIRE, is that they do 
not recognize the existence of the physical variability in 
systems components encountered in the manufacturing 
process. Rather, such simulators assume that every 
component has zero variability. However, ignoring the 
possibility of component variability in manufacturing 
leads to worst- case component design and resulting 
cost escalation. This is specially true in the case of 
flexible design and manufacturing runs where there is no 
time for use of a "learning curve" approach toward 
redesign for improved performance. 

Introducing statistical manufacturing methodology into 
the system's design simulation can greatly reduce the 
potential waste of manufacturing resources, STADIUM 
inherently recognizes that all components have natural 
variability and are not the "idealized" precision 
components typically assumed in design. It also 
recognizes that these natural variabilities feed through 
the system design in varying degrees thus affecting the 
performance of the final manufactured system. Finally, 
STADIUM greatly aids the process of nominal design 
value versus manufacturing variability trades by reducing 
requirements for precision components. 

In general, STADIUM makes full use of existing design 
simulators to avoid developing and revalidating statistical 
versions of existing system design simulators - a 
process that can be costly in terms of time and 
schedule. For instance, the FLIR90/ACQU1RE system 
design simulator was simply incorporated intact into 
STADIUM. Since the underlying simulator was not 
changed, there was also no need for any revalidations. 
The resulting statistical FLIR manufacturability 
simulation package is called STADIUM/FLIR90. Its top 
level user interface is shown below as figure 2, 
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Figure 2: The Top Level User Interface 

When the engineer completes an Input Specification, the 
Input Specification window appears as shown in Figure 
3. This Input window, provides the manufacturing 
engineer 




Figure 3: A FLIR Parameter Input Window 

with a simplified and interactive workstation display 
which is a great usability improvement over the 
conventional batch card deck input of 
FLIR90/ACQUIRE. It also permits the engineer to 
interactively specify expected standard manufacturing 
deviations of individual parameters in addition to their 
customary nominal design values. 

The Design Methodology window of STADIUM/FUR90, 
allows a non-statistician manufacturing engineer to 
straightforwardly set up manufacturing Design of 
Experiment analyses. This sets into motion a statistical 
manufacturability simulation out putting statistically 
consistent system wide results stemming from the 
manufacturing variabilities introduced in the Input 
Specification window. Alternately, STADIUM allows the 
system engineer to conduct "what if manufacturability 
sensitivity optimization experiments against specific 
series of variations. 

From the Design Methodology window, STADIUM 
automatically conducts the specified statistical 
experiment in an execution-efficient manner using the 
FLIR90/ACQUIRE simulator. 

Through the Statistical Analysis window, the system 
engineer is given a statistical analysis of system 
performance. To accomplish this analysis, STADIUM 
has the capability of either using an internal analysis 
package or if preferred an alternate drop-in third-party 
statistical analysis package such as RS/ 1 . The output is 
presented as a numerical display of nominal system 
level results (similar to the fixed output of the previous 
non-statistical system design runs ol the stand-along 
FLIR907 ACQUIRE simulator), as well as their standard 



deviations. STADIUM also provides the output 
percentage contribution contributed by each input 
component's manufacturing variation, and finally a 
mathematical model of the statistical manufacturing 
process. 

Because manufacturing engineering is multi-disciplined 
and sophisticated, STADIUM also provides a highly 
useful graphical results plotting capability. The Graphics 
plotting window of Figure 4 provides for four classes of 
graphics plotting of results, with at least 100 variations. 
These graphics can be shown at the workstation and/or 
exported for presentation quality hardcopy. 
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Figure 4: Statistical MRTD Results Curve 

For instance, Figure 4 shows a typical plot of the 
statistical manufacturability of the FLIR system's 
MRTD(Minimal Resolvable Temperature Difference). 
The variability bars show the one standard deviation 
variability at user selected critical points. 

The dotted variability envelopes show the expected 
variability over the entire MRTD range and may be used 
to insure that the entire manufacturing run will meet 
customer dictated system specifications. 

Conclusions 

A software program called STADIUM has been recently 
developed at Florida Institute of Technology to 
addresses the design issues engendered during system 
optimization against statistical variabilities. Specifically, 
STADIUM is used to predict the high-level variations of 
component arid system level parameters due to the tow- 
level variations of manulacturing and operational 
processes. STADIUM accomplishes this by setting up a 
statistically relevant design of experiment in conjunction 
with industry-standard component and system level 
simulators. Next, this statistical experiment is 



automatically run by STADIUM using the appropriate 
component and system level simulators. STADIUM then 
analyzes the results of the experiment under user 
control to provide a precise statistical description of 
component and systems performances as a function of 
manufacturing parameter variabilities. These statistical 
performance statements can then be reliable used by 
design engineers in optimizing component performance, 
reducing manufacturing costs and making life-cycle 
trade-off. 

The following are some of the key characteristics of 
STADIUM that makes it so relevant to the realm of rapid 
turn around system design. Its X-window user interface 
allows simple and rapid specification of current 
manufacturing variables and operational parameters as 
well as their variabilities. Then using appropriate 
preexisting industry simulators and its own automated 
design of experiments, STADIUM automatically runs the 
minimum number of simulations required to predict 
overall statistical performance. Thus STADIUM rapidly 
adapts to the ever changing conditions of specific 
flexible system optimization frameworks due to its 
acceptance of statistical rather than hard knowledge of 
manufacturing variabilities, and its rapid turn around 
execution times. 

STADIUM has customarily been used for statistical 
simulation of CMOS and Bipolar semiconductor 
manufacturing processes. With support from Texas 
Instruments, STADIUM has also begun to allow 
statistical analyses of sophisticated FLIR imagery sensor 
systems. Here STADIUM is capable of tracking global 
FLIR sensor performance as a user selectable function 
of over fifty manufacturing variables. Currently, 
STADIUM is being intermeshed with a Design Advisor 
program under a DARPA contract with Florida Tech as a 
subcontractor to Texas Instruments. The resulting 
"STADIUM/MADE* integrated system design 
analysis/statistical optimization tool will possess a 
coherent system wide ability to capture, synthesize, 
analyze, and model complex system designs subjected 
to a wide spectrum of statistical concerns, including 
those resulting from flexible manufacturing and 
operational use. 

Much of the expanding success of the STADIUM 
package arises from its straightforward ability to 
integrate off the shetf simulators from a variety of 
vendors. This has been an essential key allowing its 
rapid transition from semiconductor to image sensor 
domain for instance. Such changes only require making 
minor changes to the Input Specification, Data Selection 
and Graphic Display window functions of STADIUM. In 
the past, this has been done by reprogramming within 
STADIUM, but a goal of the MADE effort is to allow the 
user him or herself to more easily insert new simulators. 
This of course will greatly facilitate the rapidity with 
which flexible system optimization functions can be 
readjusted to changing market considerations. 
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Abstract 

A critical issue in the optimization of systems is that of the design of the part, component or system for manufacture. The 
system engineering function typically finds an acceptable but theoretical design solution with little up-front attention being 
paid to the "real-world" effects caused by manufacturing and operational variabilities. Alternatively when system design 
attention is paid to such variabilities, it is usually via a "worst-case" design process which can only further complicate the 
manufacturing process and raise costs. To be truly effective in the near future of small-lot flexible system manufacturing, 
systems engineering must not only optimize against the effects of changing manufacturing variabilities, but the overall 
systems design and manufacturing processes must be woven more closely together so as to permit routine first pass 
success. As a result of the above issues, the importance of running early statistically based system level design 
simulations has become particularly critical in this period of increasingly flexible manufacturing processes, defense 
conversions and dual-use technologies 
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